Click and then enter a new name for the selected album.
OR
Right click the Album Explore Pane, in the pop up menu select Rename .